Node v18.15.0 and higher ignore --max-http-header-size #47246
Closed
Description
Version
v18.15.0 and v19.8.1
Platform
Linux FESK-LTP0007 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux (Ubuntu 22.04)
Subsystem
No response
What steps will reproduce the bug?
I use urql in my NextJS application, to communicate with backend CMS. For some query CMS send extra large headers with response (around 22kb).
I use node 18 as runner for project where I use URQL library to GraphQL request to backend CMS, but there I get console error:
[newro-15509] Network Error! CombinedError: [Network] fetch failed
at makeErrorResult (file:///data/***/node_modules/@urql/core/dist/urql-core-chunk.mjs:284:10)
at file:///data/***/node_modules/@urql/core/dist/urql-core-chunk.mjs:462:15 {
graphQLErrors: [],
networkError: TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11457:11) {
cause: HeadersOverflowError: Headers Overflow Error
at Parser.trackHeader (node:internal/deps/undici/undici:9453:37)
at Parser.onHeaderValue (node:internal/deps/undici/undici:9448:14)
at wasm_on_header_value (node:internal/deps/undici/undici:9257:34)
at wasm://wasm/00036ac6:wasm-function[49]:0xf91
at wasm://wasm/00036ac6:wasm-function[68]:0x7316
at wasm://wasm/00036ac6:wasm-function[67]:0x154d
at wasm://wasm/00036ac6:wasm-function[21]:0x552
at Parser.execute (node:internal/deps/undici/undici:9373:26)
at Parser.readMore (node:internal/deps/undici/undici:9352:16)
at Socket.onSocketReadable (node:internal/deps/undici/undici:9660:14) {
code: 'UND_ERR_HEADERS_OVERFLOW'
}
},
response: undefined
}
I investigate and I found solution to set http header size by using environment: NODE_OPTIONS=--max-http-header-size=32768
This seems to be ignored for Node version 18 and higher (I also try node 19). On Node 16, this environment variable solve error.
How often does it reproduce? Is there a required condition?
No response
What is the expected behavior? Why is that the expected behavior?
No response
What do you see instead?
I see error message:
[newro-15509] Network Error! CombinedError: [Network] fetch failed
at makeErrorResult (file:///data/***/node_modules/@urql/core/dist/urql-core-chunk.mjs:284:10)
at file:///data/***/node_modules/@urql/core/dist/urql-core-chunk.mjs:462:15 {
graphQLErrors: [],
networkError: TypeError: fetch failed
at Object.fetch (node:internal/deps/undici/undici:11457:11) {
cause: HeadersOverflowError: Headers Overflow Error
at Parser.trackHeader (node:internal/deps/undici/undici:9453:37)
at Parser.onHeaderValue (node:internal/deps/undici/undici:9448:14)
at wasm_on_header_value (node:internal/deps/undici/undici:9257:34)
at wasm://wasm/00036ac6:wasm-function[49]:0xf91
at wasm://wasm/00036ac6:wasm-function[68]:0x7316
at wasm://wasm/00036ac6:wasm-function[67]:0x154d
at wasm://wasm/00036ac6:wasm-function[21]:0x552
at Parser.execute (node:internal/deps/undici/undici:9373:26)
at Parser.readMore (node:internal/deps/undici/undici:9352:16)
at Socket.onSocketReadable (node:internal/deps/undici/undici:9660:14) {
code: 'UND_ERR_HEADERS_OVERFLOW'
}
},
response: undefined
}
Additional information
No response
Activity
bnoordhuis commentedon Mar 25, 2023
Can you include a test case? No third-party modules, just built-in modules.
I'm reasonably sure the option gets passed to undici (the new http library) because we have tests that exercise the flag.
havran commentedon Mar 25, 2023
Ok, I just create wery simple HTTP server:
I run server
node server.jsand in another window I runNODE_OPTIONS=--max-http-header-size=32768 nodeand on prompt I use simple fetch request:Node version used:
os
havran commentedon Mar 25, 2023
I also attach image where I test
node --max-http-header-size=32768, seems it is works, http.maxHeaderSize is set correct, but fetch request still showUND_ERR_HEADERS_OVERFLOW:climba03003 commentedon Mar 25, 2023
undicidoes not usehttp.maxHeaderSizeas the default.It has it's own default value and you need to change it yourself.
https://github.com/nodejs/undici/blob/e6fc80f809d1217814c044f52ed40ef13f21e43c/lib/client.js#L225
You can change the behavior by
Full code that would work for your usage.
bnoordhuis commentedon Mar 26, 2023
@nodejs/http if βοΈis true, isn't that a bug?
mcollina commentedon Mar 26, 2023
Given it's a read-only value, I think undici can use that value as the default when creating the default Agent.
Shall we move this issue to undici?
havran commentedon Mar 26, 2023
IMHO this behaviour should be same than in older node versions. Also when user use frameworks like NextJS, then is unexpected when there is no simple way change this value.
adoprog commentedon Aug 23, 2023
Just got this error in NextJS after upgrading to its recent version. Is there any way to apply the workaround to it?
balazsorban44 commentedon Aug 31, 2023
My two cents here, @mcollina's #47246 (comment) makes a lot of sense to me. In newer versions of Node.js, I consider
undicias an implementation detail ofglobalThis.fetch, and any Node.js flag should configurefetchaccordingly.I don't mind if it's a different flag, even something like
--undici-max-http-header-size, but it should be configurable without importingundici.Ethan-Arrowood commentedon Aug 31, 2023
For this particular problem, users can use
setGlobalDispatcherto modify this value and it should work. But I agree that since that method is not documented well; undici should start respecting more of Node's http flags/options.It seems @mcollina is also in agreement; so lets ship it π’
climba03003 commentedon Aug 31, 2023
I believe it just the matter of will someone willing to open PR to update
undicidefault value tohttp.maxHeaderSize.--max-http-header-sizeNode.js flag nodejs/undici#2234balazsorban44 commentedon Oct 14, 2023
Hi everyone, this has shipped in Node.js, you can upgrade, and will be able to set this flag, I believe the issue can be closed @havran.